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Abstract 

Recently perfectly secure steganographic systems have been described 
for a wide class of sources of covertexts. The speed of transmission of 
secret information for these stegosystems is proportional to the length of 
the covertext. In this work we show that there are sources of covertexts 
for which such stegosystems do not exist. The key observation is that 
if the set of possible covertexts has a maximal Kolmogorov complexity, 
then a high-speed perfect stegosystem has to have complexity of the same 
order. 



1 Introduction 

The goal of steganography can be described as follows. Alice and Bob can 
exchange messages of a certain kind (called covertexts) over a public channel. 
The covertexts can be, for example, a sequence of photographic images, videos, 
text emails and so on. Alice wants to pass some secret information to Bob 
so that Eve, the observer, cannot notice that any hidden information is being 
passed. Thus, Alice should use the covertexts to hide the secret text. 

Cachin [1] suggested an information-theoretic model for steganography, along 
with a definition of a perfectly secure steganographic system. According to this 
model, Alice has an access to a probabilistic source of covertexts. It is usually 
assumed that the secret message can be represented as a sequence of indepen- 
dent equiprobable bits. She has to embed her secret message into covertexts in 
such a way that Bob can decode the message. A stegosystem is called perfectly 
secure if the distribution of the output is the same as the distribution at the 
source of covertexts. Indeed, in this case nobody is able to distinguish containers 
with hidden information and "empty" ones (i.e. without hidden information). 

Let us consider an example of a perfect stegosystem suggested in [5]. The 
source of covertexts /i is as follows. It generates sequences of n independently 
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identically distributed letters from some finite alphabet A, where n € N is given. 
For the sake of simplicity we consider the binary alphabets A — {a, b}, but the 
construction can be used for the general case too [5] . The distribution \i can be 
unknown to Alice and Bob. Suppose that Alice has to transmit a secret sequence 
V* = ViV2 ■ ■ ■ generated by a source to, where u>(yi = 0) = 1/2 independently for 
all i e N, and let there be given a covertext sequence x* = x\Xi - ■ ■ generated 
by (i. For example, let 

y* = 01100 . . . , x* = aababaaaabbaaaaabb (1) 

The sequences x* and y* are encoded in a new sequence X (to be transmitted 
to Bob) such that y* is uniquely determined by X, and the distribution of X 
is the same as the distribution of x* (that is, ^; in other words, X and x* are 
statistically indistinguishable) . 

The encoding is carried out in two steps. First we group all symbols of x* 
into pairs, and denote 

aa = u, bb = u, ab = Vq, ba = V\. 

In our example, the sequence ([T]) is represented as 

x* = aa ba ba aa ab ba aa aa bb — uv\ViuvqV\uuu 

Then X is acquired from x* as follows: all pairs corresponding to u are left 
unchanged, while all pairs corresponding to Vk are transformed to pairs corre- 
sponding to v Vl Vy 2 Vy 3 ; in our example 

X = aa ab ba aa ba ab aa aa bb 

Decoding is obvious: Bob groups the symbols of X into pairs, ignores all occur- 
rences of aa and bb and changes ab to and ba to 1. 

The described stegosystem has the following properties. The sequence of 
symbols output by the stegosystem obeys the same distribution /i as the input 
sequence. The average length of the transmitted secret sequence is n/j,(ab); in 
other words, the speed of transmission of hidden information is /i(ab) secret 
bits per letter of covertext. Moreover, in [5J a generalization of the described 
construction is proposed, for which the speed of transmission of secret text 
approaches the Shannon entropy h(fi) = —(fi(a) log /j,(a) + fi(b) log/x(6)) when 
n goes to infinity. In addition, a similar construction is proposed in the same 
work for the case of arbitrary alphabets and for finite-alphabet Markov sources 
of covertexts. 

So, we can see that perfectly secure stegosystems exist for a wide class of 
covertexts and, moreover, such systems are quite simple and have a high speed 
of transmission of secret information. Naturally, one is interested in the ques- 
tion of whether such stegosystems exist for any possible source of covertext. 
This problem is of interest since sources of covertexts that are of particular 
practical importance, such as texts in natural languages or photographs, do not 
seem to be well-described by any known simple model. Here we answer this 
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question in the negative. More precisely, it turns out that there exists such a 
set of covertexts of length n for which simple stegosystems which have speed of 
transmission of hidden text O(n) do not exist. Here simplicity is measured by 
Kolmogorov complexity of the system and a stegosystem is "simple" means that 
its complexity is exp(o(n)), when n goes to infinity. Kolmogorov complexity is 
an intuitive notion that often helps to establish results that help to understand 
the principled limitations a certain problem or model imposes; it has been used 
as such in many works, see, for example, [3J [51 ITU], 

This result can be interpreted as that there are such complicated sources 
of data, that one cannot conceivably put significantly more information into 
a source, without changing its characteristics, even though the entropy of the 
source is very high. This may explain what is known in practice; for example, 
it is apparently very hard to put any hidden message into a given text in a 
natural language, without making the text "unnatural" . Of course, rather than 
trying to change a given text, the communicating parties can easily agree in 
advance on two texts each of which codes one secret bit, so that when the need 
for communication arises, Alice can transmit one of the texts thereby passing 
one bit. However, in order to communicate more than one bit, to use the same 
method they would have to have a database of covertexts that is exponentially 
large with respect to the message to pass. Moreover, even this stegosystem will 
not be perfectly secure, since the source of covertexts with hidden information 
is concentrated on a small subset of all the possible covertexts of the given 
length. If the stegosystem is used once, then perhaps no reliable detection of 
the hidden message is possible. If it is to be used on multiple occasions, that 
is, if we wish to construct a general purpose stegosystem for transmitting, say, 
5n bits with an n-bit message (for some fixed 5), we will need to construct a 
database of effectively all possible covertexts. At least, this is the case for some 
sources of covertexts, as the result of this work demonstrates, and it seems 
likely that it is the case for such sources as texts in natural languages or even 
photographic images. Thus, our result may be helpfull in clarifying the nature 
of the difficulties that arise in construction of real stcganographic systems which 
use human-generated sources of covertexts. 

2 Preliminaries 

We use the following model for steganography, mainly following [T]. It is as- 
sumed that Alice has an access to an oracle which generates covertexts accord- 
ing to some fixed but unknown distribution of covertexts /x. Covertexts belong 
to some finite alphabet A. For the sake of simplicity we consider the case 
A = {0, 1}; the general case is analogous. Alice wants to use this source to 
transmit hidden messages. A hidden (or secret) message is a sequence of letters 
from {0, 1} generated independently with equal probabilities of and 1. We de- 
note the source of hidden messages by to. This is a commonly used model for the 
source of secret messages, since it is assumed that secret messages are encrypted 
by Alice using a key shared only with Bob. If Alice uses the Vernam cipher (a 
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one-time pad) then the encrypted messages are indeed generated according to 
the Bernoulli 1/2 distribution, whereas if Alice uses modern block or stream ci- 
phers then the encrypted sequence "looks like" a sequence of random Bernoulli 
1/2 trials. (Here "looks like" means indistinguishable in polynomial time, or 
that the likeness is confirmed experimentally by statistical data, see, e.g. [1].) 
The third party, Eve is a passive adversary: Eve is reading all messages passed 
from Alice to Bob and is trying to determine whether secret messages are being 
passed in the covertexts or not. Clearly, if covertexts with and without hidden 
information have the same probability distribution (fx) then it is impossible to 
distinguish them. 

Since the number of possible covertexts x in the set A n is finite, using a 
stegosystem once Alice can only transmit a finite number of bits of the secret 
message. We tacitly assume that there are always more secret bits than Alice 
wants to pass, which is formalized by assuming that Alice has an infinite secret 
message (in practice, if Alice runs out of secrets, she can fill the remainder of the 
message with random noise). Depending on the covertext that Alice has and 
on the actual secret message, the length of the secret text that she transmits 
may vary. Naturally, one wishes to maximize the expected length of the secret 
message that the encoder can transmit. We require, however, that the decoding 
is always correct, that is, Bob gets the whole secret message that Alice has 
transmitted, without errors. 

The steganographic protocol can be summarized in the following definitions. 

Definition 1 (secret or hidden text). A source lo of secret text y* = yi,y%, ■ ■ ■ 
is such that to(yi = 0) = uj(yi = 1) = 1/2, independently for all i 6N. 

Definition 2 (stegosystem). A stegosystem St is a family (indexed by n) of 
pairs of functions: the encoder, that maps a pair (x,y*) G A n x {0,1}°° (a 
covertext and a secret sequence) into a pair (f, StEnc n (x, (y\ . . -yt)) G N x A 11 : 
the number of secret bits transmitted and the output covertext. The decoder 
StDec n is a function from A n to {0, 1}*. We will often omit the parameter n 
from the notation, when its value is clear. 

Definition 3 (steganographic protocol). A parameter n is fixed. Alice draws a 
covertext x 6 A n generated by a source of covertexts fi (a distribution on A n ) 
and a secret message y* = J/i,y2>- •• according to the source lo. The sources to 
and fj, are independent of each other. 

Given x G A n and y* Alice using a stegosystem St obtains the number of 
secret bits she can pass t(x,y*) > 0, and a covertext StEnc(cc, (y% . . .yt)) G A n 
that is transmitted over a public channel to Bob. (Only StEnc(a;, (y\ . . .y t )) is 
transmitted; the number t is not.) 

Bob (and any possible observer Eve) receives x' G A n and obtains using the 
decoder StDec the resulting message StDec„(a;') = y\ . . . y t . 

Definition 4 (perfect security). A steganogrpahic system is called perfectly 
secure if the sequence of covertexts x* and the steganographic sequence X have 
the same distribution: Pr /jXW (StEnc = x') = fJ.(x') for any x' G A n , where the 
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first probability is taken with respect to the distribution of covertexts fj, and that 
of secret text ui. 

Definition 5 (speed of transmission). For a stegosystem St the speed of trans- 
mission of secret text u n (St) is defined as ~E fJ _ XLJ t(x,y*)/n (the expectation is 
with respect to \x and w). 

Note that often (in particular, in [T]) more general steganographic protocols 
are considered, allowing for non-perfect security, transmission with errors, sev- 
eral draws from the source of covertexts, etc. We have decided to concentrate on 
the simple model presented since it is rich enough for perfectly secure stegosys- 
tems to exist, for a wide classes of sources of covertexts (e.g. all finite-memory 
sources, [S]). Some possible extensions are discussed in Section 2] 

For definitions, notation, and an introduction to Kolmogorov complexity, 
see [3] . Informally, the Kolmogorov complexity, or algorithmic entropy, can be 
defined as follows [8]: K(x) of a string x is the length (number of bits) of a 
shortest binary program (string) to compute i on a fixed reference universal 
computer (such as a particular universal Turing machine). Intuitively, K(x) 
represents the minimal amount of information required to generate x by any 
effective process. The conditional Kolmogorov complexity K(x\y) of x relative 
to y is defined similarly as the length of a shortest program to compute x, if y 
is furnished as an auxiliary input to the computation. 

We will use some simple properties of K, such as K(s) < \s\+c for any word 
s, whose proofs can be found in e.g. [3]. Here it is worth noting that K(s) does 
not take into account the time or extra memory it takes to compute s. 

3 Main results 

Theorem 1. For every 6 > there is a family indexed by n 6 N of distri- 
butions P n on A n with h(P n ) > n — 1, such that every stegosystem St„ whose 
Kolmogorov complexity satisfies log K(St n ) — o(n) and whose speed of trans- 
mission of hidden text u„(St„) is not less than 8, is not perfectly secure from 
some n on. 

Proof. We will construct a sequence of sets X n of words of length n whose 
Kolmogorov complexity is the highest possible, namely 2™ n \ For each neN, 
the distribution P n is uniform on X n . We will then show that, in order to have 
the speed of transmission 5 > a perfectly secure stegosystem must be able 
to generate a large portion of the set X n , for each n. This will imply that the 
complexity of such a stegosystem has to be 2™ n >. The latter implication will 
be shown to follow from the fact that, in order to transmit some information, 
a stegosystem must replace the input with some output that could have been 
generated by the source; this, for perfectly secure stegosystems, amounts to 
knowing at least a large portion of X n . 

Fix n e N and let X C A n be any set such that \X\ = and 

K(X) = 2 n (l + o(l)). (2) 
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The existence of such a set can be shown by a direct calculation of the number 

of all subsets with 2™ _1 elements; the maximal complexity is equal (up to a 

constant) to the log of this value. 

Assume that there is a perfectly secure stegosystem St n for the family P n , 

n e N, and let the speed of transmission of hidden text be not less than S. 

Define the set Z as the set of those words which are used as codewords Z := 

{x E A n : StDec(x) ^ A}. Since the expected speed of transmission of hidden 

text is lower bounded by 5, we must have \Z\ > 82 n ~ 1 (indeed, since every 

word codes at most n — 1 bits, the expected speed of transmission must satisfy 
I z\ 

(n — lj^ki > Sn). Since St is perfectly secure Z C X. Let us lower-bound the 
complexity K(Z\X\Z) of the set Z given X\Z. Given the description of X\Z 
and the description of Z relative to X\Z one can reconstruct X . That is why 
K(Z\X\Z) > K{X) - K(X\Z) + 0(1). The size of X\Z is not greater than 
2"~ 1 (1 - S). Hence, 

K(Z\X\Z) > K(X) - max K(U) + 0(1). (3) 

|[/|<2"-i(l-<5) 

The latter maximal complexity can be calculated as follows: 

/ 2™ \ 
max K(U)=log[ n .J +0(1). 

|j/|<2«-i(i-<5) v ' \2 n - !(1 - 6) J 

Applying the Stirling approximation for factorial we obtain 
max KOJ) < 2™(1 - 7 )(1 + o(l)), 

|!7|<2"- 1 (l-<5) 

where 7 = 1 - h(^, From this equality, © and © we get 

K{Z) > 7 2 n (l + o(l)). 

Furthermore, define Z$ as the set of words that code those secret messages 
that start with 0, and Z\ those that start with 1: 

Zi := {x G A n : StDec(ir) =iu,ue {0, 1}*}, i G {0, 1}. (4) 

Clearly, Z = Z X UZ Q . Hence, K(Z\X\Z) < K[Z \X\Z) + K(Z t \X\Z) + 0(1), 
so that K(Zi\X\Z) > K{Z\X\Z)/2 + 0{\) for some % e {0,1}. Let this i be 1. 
Thus, 

K(Zx\X\Z) > 7 2"- 1 (l + o(l)). (5) 

We will next show how to obtain Z\ from Z\Z\ and the stegosystem St, thus 
arriving at a contradiction with the assumption that logA"(St) = o(n). 
For a set T C X define 

tp(T) := {StEnc(a;, lu):xeT,ue {0, 1}*}. 

Since St is perfectly secure, (p(T) C X for every T C X. Let T = X\Z\, and 
Tk — Tk-\ U Lp(Tk-\). Since X is finite and each T^-i is a subset of T^, there 
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must be such kg G N that = T^ a for all k > ko. There are two possibilities: 
either Tfc = X or X\Tk ^ 0. Assume the latter, and define Z{ = X\Tfc . 
Then to obtain an element of Z[ as an output of the stegosystem St, the input 
must be an element of Z[ and a secret message that starts with 1. From this, and 
from the fact that the distribution of the output is the same as the distribution 
of the input (that is, St is perfectly secure), we get 

P n (Z[) = P„(Z[,y= lu) = P n (Z[)u(l) = P„(Z[)/2, 

which implies P n (Z[) = and Z[ = 0. Therefore, there is a k G N such that 
Tk = X. This means that a description of Z\ can be obtained from a description 
of X\Z\ — T and St. Indeed, to obtain Z\ it is sufficient to run StEnc on all 
elements of To with all inputs starting with 1, thus obtaining T\, and then 
repeat this procedure until we get T^+i = Tk for some k, wherefrom we know 
that T k = X and Z x = T k \T a . Thus, 

KiZ^XXZx) < K(St) + 0(1) = 2°< n > (6) 

which contradicts ©. □ 



4 Possible extensions 

The definitions of stegosystems and steganographic protocol that we have used 
allow for several extensions. In particular, we have made the requirement that 
Alice can draw only one covertext from the source, in order to construct her 
message. We have also required that the decoding is always correct, did not 
allow for a secret key in the protocol (a secret key could be used before entering 
into steganographic communication in order to obtain the secret message y* , 
but is out of scope of the protocol), etc. These requirements, along with the 
requirement of perfect security, might be considered restrictive; however, as was 
mentioned in the Introduction, for some sources of covertexts (such as i.i.d. or 
finite-memory sources) there are indeed perfectly secure steganographic systems 
that meet all the requirements we have made, and which also have the highest 
possible speed of transmission of hidden text: u„(St) approaches (exponentially 
fast) the Shannon entropy h(fi) of the source of covertexts, as n grows (see [5]). 
This is why we have decided to sacrifice the generality for the sake of simplicity 
of the model presented. 

Nevertheless, it is worth noting that the main results of this work can be 
extended to more general cases. For example, if we allow Alice and Bob to 
share a secret key k ni then trivially Theorem 1 holds with K(St n ) replaced 
by K(St n ) + K(k n ). Let us briefly stop on another extension of the protocol. 
Instead of allowing Alice to draw only one covertext from the source, we can 
allow her to draw several, say, M covertexts. Given M covertexts x\ 1 . . . ,x m , 
where Xi S A", and a secret sequence y* Alice constructs a single x' G A n 
which is passed (over a public channel) to Bob. In particular, depending on the 
message y*, Alice can chose Xj that already encodes the message, if such x^, 
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1 < i < M exists. The speed v n of transmission of secret text is measured with 
respect to what is passed over the public channel only (i.e. x'). Then Theorem[T] 
admits the following extension: there are such sources of covertexts, that any 
perfectly secure simple stegosystem must draw M n = 2 n ( n > covertexts in order 
to transmit 5n bits, for any given S > 0. 
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